Diagnosis supporting device and method of supporting diagnosis

ABSTRACT

A display, imaging units that capture a subject, an eye gaze detector that detects an eye gaze direction of the subject, from the captured image captured by the imaging units, a gaze point detector that detects a gaze point of the subject in a display region of the display, based on the eye gaze direction, an output controller that displays any of a plurality of diagnosis images on the display, selects the diagnosis image to be displayed next according to a position of the gaze point detected when the diagnosis image is displayed, from the plurality of diagnosis images, and further displays the selected diagnosis image on the display, and an evaluator that calculates an evaluation value of the subject, based on the gaze point detected by the gaze point detector when the diagnosis image is displayed.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of PCT international application Ser.No. PCT/JP2014/069978 filed on Jul. 29, 2014 which designates the UnitedStates, incorporated herein, by reference, and which claims the benefitof priority from Japanese Patent Application No. 2013-159941, filed onJul. 31, 2013 and Japanese Patent Application No. 2014-036635, filed onFeb. 27, 2014, incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a diagnosis supporting device and amethod of supporting diagnosis.

2. Description of the Related Art

In recent years, it is said that the number of developmentally disabledpersons is increasing. The developmental disorder is known that symptomscan be relieved by early discovery and start of rehabilitation, and aneffect to adapt to society can become high. Our country has beenstriving to early discovery by a medical interview at the time ofmedical examination of 1 and a half years old, and the like. However,there are problems such as psychiatrist shortage and taking time for themedical interview, and an effect thereof cannot be said to besufficient. Therefore, an objective and efficient diagnosis supportingdevice of the developmental disorder has been sought.

For the early discovery of the developmental disorder, it is ideal tohave diagnosis at the time of the medical examination of 1 and a halfyears old, for example. An example of a characteristic of adevelopmentally disabled child includes not making eye contact with afacing person (looking away of eye gaze). Further, the developmentallydisabled child is known to have a preference for geometrical patternpictures to person pictures.

Further, methods of supporting diagnosis of developmental disorder byapplication of methods of capturing a face of a human with a camera anddetecting a gaze point by calculation of positions of a cornealreflection and a pupil have been proposed.

However, a subject of about 1 and a half years old has limited power ofconcentration. Therefore, it is necessary to perform diagnosis asbriefly as possible. However, the conventional methods of detecting agaze point may not be able to support the diagnosis in a short time, andmethods of supporting diagnosis in a shorter time have been sought.

Therefore, there is a need for a diagnosis supporting device and amethod of supporting diagnosis that enable diagnosis in a short time.

SUMMARY OF THE INVENTION

It is an object of the present invention to at least partially solve theproblems in the conventional technology.

A diagnosis supporting device includes a display, an imaging unitconfigured to capture a subject, an eye gaze detector configured todetect an eye gaze direction of the subject, from a captured imagecaptured by the imaging unit, a gaze point detector configured to detecta gaze point of the subject in a display region of the display, based onthe eye gaze direction, an output controller configured to display anyof a plurality of diagnosis images on the display, select the diagnosisimage to be displayed next according to a position of the gaze pointdetected when the diagnosis image is displayed, from the plurality ofdiagnosis images, and display the selected diagnosis image on thedisplay, and an evaluator configured to calculate an evaluation value ofthe subject, based on the gaze point detected by the gaze point detectorof when the diagnosis image is displayed.

The diagnosis supporting device and the method of supporting diagnosisaccording to the present invention exert an effect to perform diagnosisin a short time.

The above and other objects, features, advantages and technical andindustrial significance of this invention will be better understood byreading the following detailed description of presently preferredembodiments of the invention, when considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an arrangement of adisplay, stereo cameras, and a light source used in a first embodiment.

FIG. 2 is a diagram illustrating an outline of functions of a diagnosissupporting device of the first embodiment.

FIG. 3 is a block diagram illustrating an example of detailed functionsof respective units illustrated in FIG. 2;

FIG. 4 is a diagram illustrating an example of detection of an eye and adistance of when two cameras are used.

FIG. 5 is a diagram illustrating examples of diagnosis images.

FIG. 6 is a diagram illustrating an example of a time chart of diagnosisimages.

FIG. 7 is a flowchart illustrating an example of diagnosis supportingprocessing of the first embodiment.

FIG. 8 is a diagram illustrating an example of a time chart of diagnosisimages.

FIG. 9 is a diagram illustrating another example of a time chart ofdiagnosis images.

FIG. 10 is a diagram illustrating an example of an arrangement of adisplay, stereo cameras, an infrared light source of a secondembodiment.

FIG. 11 is a diagram illustrating an example of an arrangement of thedisplay, the stereo cameras, the infrared light source of the secondembodiment, and a subject.

FIG. 12 is a diagram illustrating an outline of functions of a diagnosissupporting device.

FIG. 13 is a block diagram illustrating an example of detailed functionsof respective units illustrated in FIG. 12.

FIG. 14 is a diagram illustrating an outline of processing executed bythe diagnosis supporting device of the second embodiment.

FIG. 15 is an explanatory diagram illustrating a difference between amethod of using two light sources and the second embodiment using onelight source.

FIG. 16 is a diagram for describing calculation processing ofcalculating a distance between a pupil center position and a cornealcurvature center position.

FIG. 17 is a flowchart illustrating an example of the calculationprocessing of the second embodiment.

FIG. 18 is a diagram illustrating a method of calculating a cornealcurvature center position using a distance obtained in advance.

FIG. 19 is a flowchart illustrating an example of eye gaze detectionprocessing of the second embodiment.

FIG. 20 is a diagram for describing calculation processing of amodification.

FIG. 21 is a flowchart illustrating an example of the calculationprocessing of the modification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of a diagnosis supporting device and a methodof supporting diagnosis according to the present invention will bedescribed in detail based on the drawings. Note that the presentinvention is not limited by these embodiments.

First Embodiment

Conventionally, an eye gaze of a subject to single content displayed ona monitor is detected, and developmental disorder is determinedaccording to coordinates that the subject has gazed at. However, in acase of a subject at an early age, there is a problem that it isdifficult to cause the subject to gaze at the monitor for a long timeduring diagnosis.

Therefore, a diagnosis supporting device of a first embodiment selects adiagnosis image (content) to be displayed next according to the positionof the detected gaze point, from a plurality of diagnosis images. Forexample, a plurality of indexes is provided to content (a moving imageor the like) to be viewed by the subject, and the content is virtuallydivided into a plurality of diagnosis images. Then, an unnecessarydiagnosis image is skipped, according to movement of the eye gaze of thesubject detected when a certain diagnosis image is being displayed, anda diagnosis image corresponding to the next index is displayed. In thisway, by selectively presenting a portion more suitable for diagnosissupport, diagnosis time can be reduced. As a result, a decrease in thepower of concentration can be prevented, and accuracy of the diagnosiscan be improved.

FIG. 1 is a diagram illustrating an example of an arrangement of adisplay, stereo cameras, and a light source used in the firstembodiment. As illustrated in FIG. 1, in the present embodiment, a pairof stereo cameras 102 is arranged at a lower side of a display screen101. The stereo cameras 102 are imaging units that can perform stereocapturing with infrared rays, and includes a right camera 202 and a leftcamera 204.

Infrared light emitting diode (LED) light sources 203 and 205 arerespectively arranged immediately before respective lenses of the rightcamera 202 and the left camera 204 in a circumferential direction. Theinfrared LED light sources 203 and 205 include an inner circumferentialLED and an outer circumferential LED having mutually differentwavelengths to be emitted. A pupil of a subject is detected with theinfrared LED light sources 203 and 205. As a method of detecting apupil, a method described in Japanese Laid-open Patent Publication No.2008-125619 or the like can be applied.

In detecting an eye gaze, a space is expressed in coordinates, and aposition is identified. In the present embodiment, a coordinate in an upand down direction is a Y coordinate (the up direction is +), acoordinate in a transverse direction is an X coordinate (the rightdirection is +), and a coordinate in a depth direction is a Z coordinate(the front side is +), where a middle position on the display screen 101is the origin.

FIG. 2 is a diagram illustrating an outline of functions of a diagnosissupporting device 100. FIG. 2 illustrates a part of the configurationillustrated in FIG. 1, and a configuration used for driving of theaforementioned configuration. As illustrated in FIG. 2, the diagnosissupporting device 100 includes the right camera 202, the left camera204, the infrared LED light sources 203 and 205, a speaker 105, adrive/interface (IF) 208, a controller 300, a storage 150, a display210. In FIG. 2, a positional relationship of the display screen 101 withthe right camera 202 and the left camera 204 is illustrated in aneasy-to-understand manner. However, the display screen 101 is a screendisplayed on the display 210. Note that the driver and the IF may be anintegrated body or separate bodies.

The speaker 105 functions as an audio output unit that outputs an audioand the like for prompting the subject to pay attention, at the time ofcalibration and the like.

The drive/IF 208 drives units included in the stereo cameras 102.Further, the drive/IF 208 serves as an interface between the unitsincluded in the stereo cameras 102, and the controller 300.

The storage 150 stores various types of information such as a controlprogram, a measurement result, and a diagnosis support result. Thestorage 150 stores an image to be displayed on the display 210, and thelike. The display 210 displays various types of information such as anobject image for diagnosis, and the like.

FIG. 3 is a block diagram illustrating an example of detailed functionsof the respective units illustrated in FIG. 2. As illustrated in FIG. 3,the display 210 and the drive/IF 208 are connected to the controller300. The drive/IF 208 includes camera IFs 314 and 315, an LED drivecontroller 316, and a speaker driver 322.

The right camera 202 and the left camera 204 are connected to thedrive/IF 208 respectively through the camera IFs 314 and 315. Thedrive/IF 208 drives these cameras to capture the subject.

The infrared LED light source 203 includes a wavelength 1-LED 303 and awavelength 2-LED 304. The infrared LED light source 205 includes awavelength 1-LED 305 and a wavelength 2-LED 306.

The wavelength 1-LEDs 303 and 305 perform irradiation with infrared raysof a wavelength 1. The wavelength 2-LEDs 304 and 306 perform irradiationwith infrared rays of a wavelength 2.

The wavelength 1 and the wavelength 2 are a wavelength of less than 900nm and a wavelength of 900 nm or more, respectively. When the pupil isirradiated with the infrared rays of the wavelength of less than 900 nm,and reflection light reflected at the pupil is captured, a brighterpupil image can be obtained, compared with a case where the pupil isirradiated with the infrared rays of the wavelength of 900 nm or more,and reflection light reflected at the pupil is captured. Note thatwavelengths of the infrared rays to be irradiated are not limited to theabove wavelengths, and any wavelengths may be used as long as adifference can be obtained between a result of a case where the pupil isirradiated with the infrared rays of the wavelength 1 and the reflectionlight reflected at the pupil is captured, and a result of a case wherethe pupil is irradiated with the infrared rays of the wavelength 2 andthe reflection light reflected at the pupil is captured.

The speaker driver 322 drives the speaker 105. Note that the diagnosissupporting device 100 may include an interface (printer IF) for beingconnected with a printer as a print unit. Further, the printer may beincluded inside the diagnosis supporting device 100.

The controller 300 controls the entire diagnosis supporting device 100.The controller 300 includes an eye gaze detector 351, a gaze pointdetector 352, an output controller 353, and an evaluator 354.

The eye gaze detector 351 detects an eye gaze (eye gaze direction) ofthe subject, from the captured images captured by the imaging units(stereo cameras 102). Processing of detecting an eye gaze includesprocessing of detecting a position of an eye of the subject. The gazepoint detector 352 detects a gaze point of the subject, using thedetected eye gaze direction. The gaze point detector 352 detects a gazepoint that is a point that the subject gazes at, of an object imagedisplayed on the display screen 101. As an eye gaze detection method bythe eye gaze detector 351, and a gaze point detection method by the gazepoint detector 352, any conventionally used methods can be applied.Hereinafter, a case of detecting the eye gaze direction and the gazepoint of the subject, using the stereo cameras, similarly to JapaneseLaid-open Patent Publication No. 2005-198743, will be exemplarilydescribed.

In this case, first, the eye gaze detector 351 detects the eye gazedirection of the subject, from the images captured by the stereo cameras102. The eye gaze detector 351 detects the eye gaze direction of thesubject, using the methods described in Japanese Laid-open PatentPublication No. 2005-185431 and Japanese Laid-open Patent PublicationNo. 2008-125619, and the like, for example. To be specific, the eye gazedetector 351 obtains a difference between an image obtained such thatthe pupil is irradiated with the infrared rays of the wavelength 1 andcaptured, and an image obtained such that the pupil is irradiated withthe infrared rays of the wavelength 2 and captured, and generates animage with a clarified pupil image. The eye gaze detector 351 calculatesa position of the pupil of the subject (a position of the eye) by astereoscopic technique, using two images generated as described abovefrom the images captured by the right and left cameras (the right camera202 and the left camera 204), respectively. Further, the eye gazedetector 351 calculates a position of a corneal reflection of thesubject, using the images captured by the right and left cameras. Theeye gaze detector 351 then calculates an eye gaze vector that indicatesthe eye gaze direction of the subject, from the position of the pupil ofthe subject and a corneal reflection position.

Note that a method of detecting the position of the eye and the eye gazeof the subject is not limited to the above method. For example, theposition of the eye of the subject and the eye gaze may be detected, byan analysis of an image captured using visible light, instead of theinfrared rays.

The gaze point detector 352 detects an intersection point of the eyegaze vector expressed in a coordinate system like FIG. 1, and an XYplane, as the gaze point of the subject, for example. When the gazepoint detector 352 can obtain eye gaze directions of both eyes, the gazepoint detector 352 may measure the gaze point by obtaining anintersection point of right and left eye gazes of the subject.

FIG. 4 is a diagram illustrating an example of detection of an eye and adistance of when using the two cameras (the right camera 202 and theleft camera 204). A camera calibration theory by a stereo calibrationmethod is applied to the two cameras, and a camera parameter is obtainedin advance. As the stereo calibration method, any conventionally usedmethod such as a method using a camera calibration theory of Tsai can beapplied. Three-dimensional coordinates of the eye in a world coordinatesystem can be obtained using the position of the eye detected from theimage captured by the right camera 202, the position of the eye detectedfrom the image captured by the left camera 204, and the cameraparameter. Accordingly, the distance between the eye and the stereocameras 102, and pupil coordinates can be estimated. The pupilcoordinates are coordinate values that indicate the position of the eye(pupil) of the subject on the XY plane. The pupil coordinates can becoordinate values of the position of the eye expressed in the worldcoordinate system being projected onto the XY plane, for example.Usually, the pupil coordinates of the right and left both eyes areobtained. A diagnosis image is displayed on the display screen 101.

Note that the diagnosis image may be either a still image or a movingimage (picture). However, the moving image is desirable because asubject can easily gaze at the moving image. The diagnosis image mayinclude a plurality of diagnosis images that is temporally andcontinuously displayed, for example. As described above, one picture (amoving image or the like) may be divided and used as the plurality ofdiagnosis images. Each of the plurality of diagnosis images is an imagewith which whether the subject has the developmental disorder can bediagnosed. For example, at least a part of the plurality of diagnosisimages may include an image of a person (person picture) and an image ofa geometrical pattern (geometrical pattern picture). A natural image maybe used as an image other than the geometrical pattern that the subjectwith the developmental disorder has a preference. The natural image mayjust be an image of a natural object or an image that is associated witha natural object, other than the geometrical image. For example, animage (a still image or a moving image) obtained by capturing of aperson, an animal, a plant, a landscape of nature, or the like, with acamera, may be used as the natural image. Further, an image (a stillimage or a moving image) of a character that mimics a person or ananimal may be used as the natural image.

Referring back to FIG. 3, the output controller 353 controls outputs ofvarious types of information for the display 210, the speaker 105, andthe like. For example, the output controller 353 controls outputs to thedisplay 210, such as the diagnosis image and an evaluation result by theevaluator 354. The output controller 353 displays any of the pluralityof diagnosis images on the display, and selects a diagnosis image to bedisplayed next, according to positions of a gaze point detected when thediagnosis image is being displayed, from a plurality of diagnosisimages. The output controller 353 then displays the selected diagnosisimage on the display, and repeats similar processing.

As a method of selecting the diagnosis image to be displayed next, afollowing method can be applied, for example. First, the outputcontroller 353 classifies the plurality of diagnosis images into any ofa plurality of groups. For example, the output controller 353 mayperform classification such that similar diagnosis images belong to thesame group. Further, the output controller 353 may performclassification such that a plurality of diagnosis images (a firstdiagnosis image and a second diagnosis image) in which two images (forexample, a person picture and a geometrical pattern picture) arearranged to be symmetrical to a predetermined point or line belongs tothe same group.

The output controller 353 then switches, according to the positions ofthe gaze point detected when the diagnosis image classified in a certaingroup (predetermined group) is being displayed, whether selectinganother diagnosis image classified in the same group (predeterminedgroup), or selecting a diagnosis image classified in another group(another group different form the predetermined group, of the pluralityof groups), as the diagnosis image to be displayed next. Accordingly,when it is determined that the subject has no tendency of thedevelopmental disorder for the diagnosis image belonging to the certaingroup, or the diagnosis with the diagnosis image is difficult, displayof the remaining diagnosis image of the group can be skipped.

Various conditions can be applied as a condition to perform switching.For example, a condition that another diagnosis image in the same groupis selected when the positions of the detected gaze point are matchedwith a pattern of positions of a gaze point of a subject with thedevelopmental disorder, and diagnosis image of the next group isselected in other cases may be applied.

As the pattern of positions of a gaze point of a subject with thedevelopmental disorder, a pattern of a ratio of the gaze points includedin a region of an eye included in a person picture being less than apredetermined threshold (for example, 80%) can be applied, for example.An applicable pattern is not limited to the above example, and anypattern is applicable as long as tendency of the developmental disordercan be determined from the ratio of the gaze point included in aspecific region in a predetermined period. For example, in a case of adiagnosis image that includes a person picture and a geometrical patternpicture, a pattern of a ratio of the gaze point included in the personpicture being a predetermined threshold or less may be used. The ratioof the gaze point included in a certain region (image) is obtained by,for example, the number of gaze points detected in the region (or atotal time in which the gaze point is detected within the region), tothe total number of the detected gaze points (or a total time in whichthe gaze point is detected).

The evaluator 354 calculates an evaluation value as an index related tothe degree of the developmental disorder, based on the diagnosis imageand the gaze point detected by the gaze point detector 352. Theevaluator 354 calculates the evaluation value, based on positions of thegaze point of the subject of when the diagnosis image like FIG. 5described below is displayed. For example, the evaluator 354 calculatesa ratio of looking at a predetermined region (a region of an eye, forexample) within the diagnosis image, as the evaluation value. Theevaluator 354 calculates the evaluation value that indicates thepossibility of the developmental disorder is higher as the evaluationvalue is lower. The evaluator 354 may employ an average value or a totalvalue of values calculated for a plurality of respective diagnosisimages, as a final evaluation value. The evaluator 354 may justcalculate the evaluation value, based on the diagnosis image and thegaze point, and a calculation method thereof is not limited to theembodiment.

FIG. 5 is a diagram illustrating examples of the diagnosis images. FIG.5 illustrates an example of virtually dividing one continuous movingimage into eight moving images (diagnosis images) by marking switchingpoints of the diagnosis images with marks (indexes).

A diagnosis image 501 is a still image picture of a person. Which regionof a region of an eye, a region of a mouse and the like included in thediagnosis image 501 the subject gazes at becomes an index of diagnosis(evaluation value). Typically, when the developmental disorder issuspected, the subject tends to gaze at a region other than the eye.

A diagnosis image 502 is a picture of a person talking to the subject.Similarly to the diagnosis image 501, which region of the region of aneye, the region of a mouse and the like included in the diagnosis image502 the subject gazes at becomes an index of diagnosis. Typically, whenthe developmental disorder is suspected, the subject tends to gaze at aregion other than the eye.

The left half of a diagnosis image 503 is a picture expressing a statewhere a person moves to music, using a plurality of points. The righthalf of the diagnosis image 503 is a picture upside down expressing astate where a person moves without harmonizing with music, using aplurality of points. The left-side picture is more likely to become agazing object because the movement overlaps with movement of a normalfull-length figure of a human. Therefore, frequencies of an eye gaze tothe right and left pictures become an index. Meanwhile, when thedevelopmental disorder is suspected, it is typically difficult for thesubject to recognize the group of points as movement of a person, andthere is less deviation of the gaze point.

A diagnosis image 504 is a picture in which the right and left imagesincluded in the diagnosis image 503 is interchanged. That is, thediagnosis image 503 and the diagnosis image 504 correspond to aplurality of diagnosis images arranged symmetrical to a line set inadvance (a line in the up and down direction, which passes through avicinity of the center of the diagnosis image). By use of such twodiagnosis images, whether the eye gaze of the subject depends on a rightand left positional relationship can be confirmed, and more accuratediagnosis can be performed.

A diagnosis image 505 is an diagnosis image in which a picture ofpersons is arranged on the left half, and a picture of a geometricalpattern is arranged on the right half. Frequencies of the eye gaze tothe right and left pictures become an index of diagnosis. When thedevelopmental disorder is suspected, it is typically considered that thesubject has a preference for the picture of a geometrical pattern to thepicture of persons.

A diagnosis image 506 is a diagnosis image in which a picture of ageometrical pattern is arranged on the left half, and a picture ofpersons is arranged on the right half, in an opposite manner to thediagnosis image 505. The pictures are slightly different from those ofthe diagnosis image 505. Similarly to the diagnosis image 504, thediagnosis image 506 is used to confirm whether the eye gaze position ofthe subject depends on the right and left positional relationship.

A diagnosis image 507 is a diagnosis image in which a picture of personsis arranged in the entire screen, and a picture of a geometrical patternis arranged in a partial region. Frequencies of the eye gaze to thegeometrical pattern picture and to a region other than the geometricalpattern picture become an index of diagnosis.

A diagnosis image 508 is a diagnosis image that displays a picture of ageometrical pattern in a position different from the diagnosis image507. Similarly to the diagnosis image 504, the diagnosis image 508 isused to confirm whether the eye gaze position of the subject depends onthe right and left positional relationship.

In the example of FIG. 5, the eight diagnosis images can be classifiedinto the four groups below:

Group A: the diagnosis image 501 and the diagnosis image 502

Group B: the diagnosis image 503 and the diagnosis image 504

Group C: the diagnosis image 505 and the diagnosis image 506

Group D: the diagnosis image 507 and the diagnosis image 508

Each of the groups includes similar diagnosis images used to calculate asimilar index (evaluation value). Note that the number of diagnosisimages included in each of the groups is not limited to two, and threeor more diagnosis images may be included.

For example, one of the diagnosis image 502 belonging to the same group,and the diagnosis image 503 of the next group B is selected as adiagnosis image to be displayed next, according to the position of thegaze point detected when the diagnosis image 501 of the group A is beingdisplayed. In this case, the group A is a predetermined group, and thegroup B is another group different from the predetermined group, of theplurality of groups. Similarly, one of the diagnosis image 504 belongingto the same group, and the diagnosis image 505 of the next group C isselected as a diagnosis image to be displayed next according to theposition of the gaze point detected when the diagnosis image 503 of thegroup B is being displayed. In this case, the group B is thepredetermined group, and the group C is the another group different fromthe predetermined group, of the plurality of groups. Similar processingis applied to the groups C and D.

FIG. 6 is a diagram illustrating an example of a time chart of diagnosisimages to be displayed. FIG. 6 is an example of a case where thediagnosis images 501 to 508 of FIG. 5 are displayed in order. A displaytime of each diagnosis image is 10 seconds, and the display isterminated in 80 seconds in total. To perform processing of reduction oftime, indexes (INDEXES 1 to 7) are provided at the switching points ofthe diagnosis images. During display of each diagnosis image on thedisplay 210, detection and diagnosis of the eye gaze of the subject areperformed. Therefore, normally, it is necessary to cause the subject togaze at the display 210 for 80 seconds at least. The present embodimentreduces this time by a method like below.

First, an example of reduction of diagnosis time, using a flowchart ofFIG. 7 and a time chart of FIG. 8, will be described. FIG. 7 is aflowchart illustrating an example of diagnosis supporting processing ofthe first embodiment. FIG. 8 is a diagram illustrating an example of atime chart of a diagnosis image to be displayed.

First, the output controller 353 starts reproduction of a diagnosisimage as illustrated in FIG. 5. Further, the eye gaze detector 351 andthe gaze point detector 352 starts an operation to detect the eye gazeand the gaze point (step S101). The output controller 353 displays thediagnosis image 501 of FIG. 5, first (step S102).

The output controller 353 determines whether 80% or more of gaze pointpositions of the subject detected when the diagnosis image 501 is beingdisplayed is included in the region of the eye of the person picture(step S103). Note that the threshold (80%) used for determination is anexample and is not limited to the example. Further, when the diagnosisimage is being displayed may be a part of the period in which thediagnosis image is being displayed. The same applies to the descriptionbelow.

When 80% or more of the gaze point positions is not included in theregion of the eye (No at step S103), the output controller 353 selectsthe diagnosis image 502, as the image to be displayed next, and displaysthe diagnosis image 502 (step S104). That is, when 80% or more of thegaze point positions is not included in the region of the eye, there isa possibility of the developmental disorder, and thus the outputcontroller 353 displays the similar diagnosis image 502 again.Accordingly, more accurate diagnosis can be performed. After displayingthe diagnosis image 502, the output controller 353 displays thediagnosis image 503 that is the image of the next group (step S105).

When 80% or more of the gaze point positions of the subject detectedwhen the diagnosis image 501 is being displayed is included in theregion of the eye (Yes at step S103), the output controller 353 skipsthe diagnosis image 502, selects the diagnosis image 503, as the imageto be displayed next, and displays the diagnosis image 503 (step S105).

In the example of FIG. 8, the subject has looked at the region of theeye at a rate of 80% or more. Therefore, it is determined that there isno tendency of the developmental disorder for the diagnosis image 501,or the diagnosis with the diagnosis image 501 is difficult. Therefore,it is not effective to further display the diagnosis image 502 similarto the diagnosis image 501 to diagnose the developmental disorder.Therefore, the output controller 353 skips display of the diagnosisimage 502, and jumps to a next picture position (INDEX 2). In theexample of FIG. 8, the output controller 353 jumps from the INDEX 1 tothe INDEX 2.

Next, the output controller 353 determines whether 80% or more of thegaze point positions of the subject detected when the diagnosis image503 is being displayed is included in the left half region of thediagnosis image 503 (step S106). When 80% or more of the gaze pointpositions is not included in the left half region (No at step S106), theoutput controller 353 selects the diagnosis image 504, as the image tobe displayed next, and displays the diagnosis image 504 (step S107).After displaying the diagnosis image 504, the output controller 353displays the diagnosis image 505 (step S108). When 80% or more of thegaze point positions is included in the left half region (Yes at stepS106), the output controller 353 skips the diagnosis image 504, selectsthe diagnosis image 505, as the image to be displayed next, and displaysthe diagnosis image 505 (step S108).

In the example of FIG. 8, the subject has looked at the right half by80% or more. Therefore, the diagnosis image 504 is displayed next, andmore accurate diagnosis is performed.

Next, the output controller 353 determines whether 80% or more of thegaze point positions of the subject detected when the diagnosis image505 is being displayed is included in the region of the persons of thediagnosis image 505 (step S109). When 80% or more of the gaze pointpositions is not included in the region of the persons (No at stepS109), the output controller 353 selects the diagnosis image 506, as theimage to be displayed next, and displays the diagnosis image 506 (stepS110). After displaying the diagnosis image 506, the output controller353 displays the diagnosis image 507 (step S111). When 80% or more ofthe gaze point positions is included in the region of the persons (Yesat step S109), the output controller 353 skips the diagnosis image 506,selects the diagnosis image 507, as the image to be displayed next, anddisplays the diagnosis image 507 (step S111).

In the example of FIG. 8, the subject has looked at the pattern regionby 80% or more. Therefore, the diagnosis image 506 is displayed next,and more accurate diagnosis is performed.

Next, the output controller 353 determines whether 80% or more of thegaze point positions of the subject detected when the diagnosis image507 is being displayed id included in the region of the persons of thediagnosis image 507 (step S112). When 80% or more of the gaze pointpositions is not included in the region of the persons (No at stepS112), the output controller 353 selects the diagnosis image 508, as theimage to be displayed next, and displays the diagnosis image 508 (stepS113). When 80% or more of the gaze point positions is included in theregion of the persons (Yes at step S112), the output controller 353skips display of the diagnosis image 508, and terminates the display ofthe diagnosis images.

In the example of FIG. 8, the subject has looked at the pattern regionby 80% or more. Therefore, the diagnosis image 508 is displayed next,and more accurate diagnosis is performed.

The eye gaze detector 351 and the gaze point detector 352 terminate thedetection operation of the eye gaze and the gaze point of the subject(step S114). The evaluator 354 calculates the evaluation values for therespective diagnosis images (step S115).

In the example of FIG. 8, the time to reproduce the diagnosis image 502can be omitted. Therefore, the diagnosis time can be reduced from 80seconds to 70 seconds.

Next, another example of reduction of diagnosis time, using a time chartof FIG. 9, will be described. FIG. 9 is a diagram illustrating anotherexample of a time chart of diagnosis images to be displayed.

The detection operation of the eye gaze and the gaze point of thesubject is started (step S101). The output controller 353 displays thediagnosis image 501, and determines the gaze point positions of thistime (step S103). In the example of FIG. 9, the subject has looked atthe region other than the eye by 80% or more, the diagnosis image 502 isdisplayed next, and more accurate diagnosis is performed (step S104).

Next, the output controller 353 displays the diagnosis image 503 (stepS105), and determines the gaze point positions of this time (step S106).In the example of FIG. 9, the subject has looked at the left half by 80%or more, and thus the subject has no tendency of the developmentaldisorder for the diagnosis image 503, or the diagnosis with thediagnosis image 503 is difficult. Therefore, the diagnosis with the nextdiagnosis image 504 is not effective. Therefore, the output controller353 skips display of the diagnosis image 504, and jumps to the nextpicture position (INDEX 4). In the example of FIG. 9, the outputcontroller 353 jumps from the INDEX 3 to the INDEX 4.

Next, the output controller 353 displays the diagnosis image 505 (stepS108), and determines the gaze point positions of this time (step S109).In the example of FIG. 9, the subject has looked at the region otherthan the pattern region by 80% or more, and thus the subject has notendency of the developmental disorder for the diagnosis image 505, orthe diagnosis with the diagnosis image 505 is difficult. Therefore, thediagnosis with the next diagnosis image 506 is not effective. Therefore,the output controller 353 skips display of the diagnosis image 506, andjumps to the next picture position (INDEX 6). In FIG. 9, the outputcontroller 353 jumps from the INDEX 5 to the INDEX 6.

Next, the output controller 353 displays the diagnosis image 507 (stepS111), and determines the gaze point positions of this time (step S112).In the example of FIG. 9, the subject has looked at the pattern regionby 80% or more, and thus the output controller 353 displays thediagnosis image 508, and more accurate diagnosis is performed (stepS113).

Finally, the detection operation of the eye gaze and the gaze point ofthe subject is terminated (step S114), and the evaluator 354 calculatesthe evaluation values for the respective diagnosis images.

In the example of FIG. 9, the time to reproduce the diagnosis images 504and 506 can be omitted, and thus the diagnosis time is reduced from 80seconds to 60 seconds. As described above, the reduction of thediagnosis time can be achieved by skipping of unnecessary parts of thediagnosis images in accordance with each subject.

As described above, according to the first embodiment, effects asfollows can obtained, for example.

(1) An item that enables more clear diagnosis support, in a plurality ofdetermination algorithms having different arrangements of the personpicture and the plurality of geometrical patterns, can be selectivelypresented, and the diagnosis time can be reduced.(2) By the selective presentation of the item that enables cleardiagnosis support, more accurate diagnosis support can be performed.

Second Embodiment

In a second embodiment, an eye gaze detection apparatus and an eye gazedetection method are realized, which can further simplify a deviceconfiguration compared with the first embodiment.

Hereinafter, an eye gaze detection apparatus and an eye gaze detectionmethod according to the second embodiment will be described in detailbased on the drawings. Note that the present invention is not limited bythis embodiment. Hereinafter, an example of using an eye gaze detectionapparatus, as a diagnosis supporting device that supports diagnosis ofdevelopmental disorder and the like, using eye gaze detection results,will be described. An applicable device is not limited to the diagnosissupporting device.

An eye gaze detection apparatus (diagnosis supporting device) accordingto the present embodiment detects an eye gaze, using an illuminatorinstalled in one place. Further, the eye gaze detection apparatus(diagnosis supporting device) according to the present embodimentcalculates a corneal curvature center position in a high accuratemanner, using a result of measurement obtained by causing a subject togaze at one point, before detection of the eye gaze.

Note that the illuminator is an element that includes a light source andcan irradiate an eye ball of the subject with light. The light source isan element that emits light, such as a light emitting diode (LED). Thelight source may be configured from one LED, or may be configured suchthat a plurality of LEDs is combined and is arranged at one place.Hereinafter, “light source” may be used as a term that indicates theilluminator.

FIGS. 10 and 11 are diagrams illustrating an example of arrangement of adisplay, stereo cameras, an infrared light source of the secondembodiment, and a subject. Note that the same configuration as the firstembodiment is denoted with the same reference sign, and descriptions maybe omitted.

As illustrated in FIG. 10, a diagnosis supporting device according tothe second embodiment includes a display 210, a stereo camera 2102, andan LED light source 2103. The stereo camera 2102 is arranged under thedisplay 210. The LED light source 2103 is arranged at a center positionof two cameras included in the stereo camera 2102. The LED light source2103 is, for example, a light source that irradiates the subject with anear infrared ray of wavelength 850 nm. FIG. 10 illustrates an examplein which the LED light source 2103 (illuminator) is configured from nineLEDs. Note that, in the stereo camera 2102, a lens that can transmitnear infrared light with a wavelength of 850 nm is used.

As illustrated in FIG. 11, the stereo camera 2102 includes a rightcamera 2202 and a left camera 2203. The LED light source 2103 irradiatesan eye ball 111 of the subject with the near infrared light. In an imageobtained by the stereo camera 2102, a pupil 112 is reflected at lowluminance and becomes dark, and corneal reflection 113 caused in the eyeball 111, as a virtual image, is reflected at high luminance and becomesbright. Therefore, positions of the pupil 112 and the corneal reflection113 on the image can be obtained by the two cameras (the right camera2202 and the left camera 2203).

Further, three-dimensional world coordinate values of positions of apupil 112 and a corneal reflection 113 are calculated from positions ofthe pupil 112 and the corneal reflection 113 obtained by the twocameras. In the present embodiment, as the three-dimensional worldcoordinates, a coordinate in an up and down direction is a Y coordinate(the up direction is +), a coordinate in a transverse direction is an Xcoordinate (the right direction is +), and a coordinate in a depthdirection is a Z coordinate (the front side is +), where a middleposition on a display screen 101 is an origin.

FIG. 12 is a diagram illustrating an outline of functions of a diagnosissupporting device 2100 of the second embodiment. FIG. 12 illustrates apart of the configurations illustrated in FIGS. 10 and 11, andconfigurations used for driving the aforementioned configurations. Asillustrated in FIG. 12, the diagnosis supporting device 2100 includesthe right camera 2202, the left camera 2203, the LED light source 2103,a speaker 105, a drive/IF (interface) 208, a controller 2300, a storage150, and the display 210. In FIG. 12, a positional relationship betweena display screen 101, and the right camera 2202 and the left camera 2203is illustrated in an easily understandable manner. The display screen101 is a screen displayed in the display 210. Note that the driver andthe IF may be integrated or separated.

The speaker 105 functions as an audio output unit that outputs an audioand the like for prompting the subject to pay attention, at the time ofcalibration and the like.

The drive/IF 208 drives units included in the stereo camera 2102.Further, the drive/IF 208 serves as an interface between the unitsincluded in the stereo camera 2102, and the controller 2300.

The controller 2300 can be realized by a computer that includes acontrol device such as a central processing unit (CPU), a storage devicesuch as a read only memory (ROM) and a random access memory (RAM), acommunication I/F that is connected with a network and performscommunication, and a bus that connects the units.

The storage 150 stores various types of information such as a controlprogram, a measurement result, and a diagnosis support result. Thestorage 150 stores an image to be displayed in the display 210, and thelike. The display 210 displays various types of information such as anobject image for diagnosis, and the like.

FIG. 13 is a block diagram illustrating an example of detailed functionsof the respective units illustrated in FIG. 12. As illustrated in FIG.13, the display 210 and the drive/IF 208 are connected to the controller2300. The drive/IF 208 includes camera IFs 314 and 315, an LED drivecontroller 316, and a speaker driver 322.

The right camera 2202 and the left camera 2203 are connected to thedrive/IF 208 through the camera IFs 314 and 315, respectively. Thedrive/IF 208 drives these cameras to capture the subject.

The speaker driver 322 drives the speaker 105. Note that the diagnosissupporting device 2100 may include an interface (printer IF) for beingconnected with a printer as a print unit. Further, the printer may beincluded inside the diagnosis supporting device 2100.

The controller 2300 controls the entire diagnosis supporting device2100. The controller 2300 includes a first calculator 2351, a secondcalculator 2352, a third calculator 2353, an eye gaze detector 2354, agaze point detector 2355, an output controller 2356, and an evaluator2357. Note that at least the first calculator 2351, the secondcalculator 2352, the third calculator 2353, and the eye gaze detector2354 may just be included as an eye gaze detection apparatus.

The elements (the first calculator 2351, the second calculator 2352, thethird calculator 2353, the eye gaze detector 2354, the gaze pointdetector 2355, the output controller 2356, and the evaluator 2357)included in the controller 2300 may be realized by software (a program),may be realized by a hardware circuit, or may be realized by use of thesoftware and the hardware circuit together.

When the elements are realized by the programs, the programs arerecorded in a computer-readable recording medium such as a compact diskread only memory (CD-ROM), a flexible disk (FD), a compact diskrecordable (CD-R), or a digital versatile disk (DVD) in a file in aninstallable format or in an executable format, and provided as acomputer program product. The programs may be stored on a computerconnected to a network such as the Internet, and provided by beingdownloaded through the network. Further, the programs may be provided ordistributed through the network such as the Internet. Further, theprograms may be configured to be provided by being incorporated in ROMor the like in advance.

The first calculator 2351 calculates a position (first position) of apupil center that indicates a center of a pupil, from an image of an eyeball captured by a stereo cameras 2102. The second calculator 2352calculates a position (second position) of a corneal reflection centerthat indicates a center of a corneal reflection, from the captured imageof an eye ball. The first calculator 2351 and the second calculator 2352correspond to a position detector that detects the first position thatindicates the center of the pupil and the second position that indicatesthe center of the corneal reflection.

The third calculator 2353 calculates a corneal curvature center (fourthposition), from a straight line (first straight line) that connects anLED light source 2103 and the corneal reflection center. For example,the third calculator 2353 calculates a position where a distance fromthe corneal reflection center becomes a predetermined value, on thestraight line, as the corneal curvature center. As the predeterminedvalue, a value determined in advance from a curvature radius value of atypical cornea can be used.

An individual difference may be caused in the curvature radius value ofa cornea. Therefore, there is a possibility that an error becomes largewhen the corneal curvature center is calculated using the valuedetermined in advance. Therefore, the third calculator 2353 maycalculate the corneal curvature center in consideration of theindividual difference. In this case, the third calculator 2353 firstcalculates an intersection point of a straight line (second straightline) that connects the pupil center and a target position, and straightline that connects the corneal reflection center and the LED lightsource 2103, using the pupil center and the corneal reflection centercalculated when the subject is caused to gaze at a target position(third position). The third calculator 2353 then calculates a distance(first distance) between the pupil center and the calculatedintersection point, and stores the calculated distance in a storage 150,for example.

The target position may be any position as long as the position can bedetermined in advance, and three-dimensional world coordinate values canbe calculated. For example, a middle position (the origin of thethree-dimensional world coordinates) of the display screen 101 can beused as the target position. In this case, for example, the outputcontroller 2356 displays an image (target image) or the like that thesubject is caused to gaze at, in the target position (center) on thedisplay screen 101. Accordingly, the subject can be caused to gaze atthe target position.

The target image may be any image as long as the image can drawattention from the subject. For example, an image with a varying displayform such as luminance or a color, an image having different displayform from other regions, or the like can be used as the target image.

Note that the target position is not limited to the center of thedisplay screen 101, and any position can be employed. If the center ofthe display screen 101 is employed as the target position, the distancebetween the center and any end part of the display screen 101 isminimized. Therefore, for example, a measurement error at the time ofdetecting the eye gaze can be made smaller.

The processing up to the calculation of the distance is executed inadvance before actual detection of an eye gaze is started, for example.At the time of actual detection of an eye gaze, the third calculator2353 calculates a position where the distance from the pupil centerbecomes the distance calculated in advance, on the straight line thatconnects the LED light source 2103 and the corneal reflection center, asthe corneal curvature center. The third calculator 2353 corresponds to acalculator that calculates the corneal curvature center (fourthposition), from the position of the LED light source 2103, thepredetermined position (third position) that indicates the target imageon the display, the position of the pupil center, and the position ofthe corneal reflection center.

The eye gaze detector 2354 detects the eye gaze of the subject from thepupil center and the corneal curvature center. For example, the eye gazedetector 2354 detects a direction from the corneal curvature centertoward the pupil center, as an eye gaze direction of the subject.

The gaze point detector 2355 detects a gaze point of the subject, usingthe detected eye gaze direction. The gaze point detector 2355 detects,for example, a gaze point that is a point that the subject gazes at onthe display screen 101. The gaze point detector 2355 detects anintersection point of an eye gaze vector and an XY plane, which areexpressed in a three-dimensional world coordinate system as illustratedin FIG. 11, as the gaze point of the subject.

The output controller 2356 controls output of various types ofinformation to the display 210, the speaker 105, and the like. Forexample, the output controller 2356 outputs the target image to thetarget position on the display 210. Further, the output controller 2356controls output to the display 210, such as a diagnosis image, anevaluation result by the evaluator 2357, and the like.

The diagnosis image may just be an image according to evaluationprocessing based on an eye gaze (gaze point) detection result. Forexample, when a developmental disorder is diagnosed, a diagnosis imagethat includes an image (a geometrical pattern picture or the like)preferred by the subject with the developmental disorder, and anotherimage (a picture of a person, or the like) may be used.

The evaluator 2357 performs evaluation processing based on the diagnosisimage, and the gaze point detected by the gaze point detector 2355. Forexample, when the developmental disorder is diagnosed, the evaluator2357 analyzes the diagnosis image and the gaze point, and evaluateswhether the subject with the developmental disorder has gazed at theimage that the subject prefers.

The output controller 2356 may display a diagnosis image similar to thefirst embodiment, and the evaluator 2357 may perform evaluationprocessing similar to the evaluator 354 of the first embodiment. Inother words, the eye gaze detection processing (eye gaze detector 351)of the first embodiment may be replaced with the eye gaze detectionprocessing (the first calculator 2351, the second calculator 2352, thethird calculator 2353, and the eye gaze detector 2354) of the secondembodiment. Accordingly, an effect of the second embodiment(simplification of the device configuration, and the like) can beachieved in addition to the effect of the first embodiment.

FIG. 14 is a diagram for describing an outline of processing executed bythe diagnosis supporting device 2100 of the present embodiment. Elementsdescribed in FIGS. 10 to 13 are denoted with the same reference signs,and descriptions are omitted.

A pupil center 407 and a corneal reflection center 408 respectivelyindicate the center of the pupil and the center of a corneal reflectionpoint detected when the LED light source 2103 is lighted. A corneacurvature radius 409 indicates the distance from a surface of the corneato a corneal curvature center 410.

FIG. 15 is an explanatory diagram illustrating a difference between amethod using two light sources (illuminators) (hereinafter, referred toas method A), and the present embodiment using one light source(illuminator). Elements described in FIGS. 10 to 13 are denoted with thesame reference signs, and descriptions are omitted.

The method A uses two LED light sources 511 and 512, in place of the LEDlight source 2103. In the method A, an intersection point of a straightline 515 that connects a corneal reflection center 513 and the LED lightsource 511 of when the LED light source 511 irradiates the subject withlight, and a straight line 516 that connects a corneal reflection center514 and the LED light source 512 of when the LED light source 512irradiates the subject with light is calculated. This intersection pointserves as a corneal curvature center 555.

In contrast, in the present embodiment, a straight line 523 thatconnects a corneal reflection center 522 and the LED light source 2103of when the LED light source 2103 irradiates the subject with light isconsidered. The straight line 523 passes through the corneal curvaturecenter 555. Further, the curvature radius of a cornea is known to have asmall influence due to the individual difference and have a nearly fixedvalue. According to this fact, the corneal curvature center of when theLED light source 2103 irradiates the subject with light exists on thestraight line 523, and can be calculated using a typical curvatureradius value.

However, when the gaze point is calculated using the position of thecorneal curvature center obtained using the typical curvature radiusvalue, the gaze point position is deviated from an original position dueto the individual difference of the eye ball, and an accurate gaze pointposition may not be able to be detected.

FIG. 16 is a diagram for describing calculation processing ofcalculating a corneal curvature center position, and the distancebetween a pupil center position and the corneal curvature centerposition, before the gaze point detection (eye gaze detection) isperformed. Elements described in FIGS. 10 to 13 are denoted with thesame reference signs, and descriptions are omitted. Note that connectionof the right and left cameras (the right camera 2202 and the left camera2203) and the controller 2300 is not illustrated and is omitted.

A target position 605 is a position for causing the subject to gaze at,by an output of a target image or the like to one point on the display210. In the present embodiment, the target position 605 is a middleposition on the display screen 101. A straight line 613 is a straightline that connects the LED light source 2103 and a corneal reflectioncenter 612. A straight line 614 is a straight line that connects thetarget position 605 (gaze point) that the subject gazes at and a pupilcenter 611. A corneal curvature center 615 is an intersection point ofthe straight line 613 and the straight line 614. The third calculator2353 calculates and stores a distance 616 between the pupil center 611and the corneal curvature center 615.

FIG. 17 is a flowchart illustrating an example of calculation processingof the present embodiment.

First, the output controller 2356 reproduces the target image at onepoint on the display screen 101 (step S201), and prompts the subject togaze at the one point. Next, the controller 2300 lights the LED lightsource 2103 toward an eye of the subject, using an LED drive controller316 (step S202). The controller 2300 captures the eye of the subject bythe right and left cameras (the right camera 2202 and the left camera2203) (step S203).

By the irradiation of the LED light source 2103, a pupil part isdetected as a dark part (dark pupil). Further, a virtual image ofcorneal reflection is caused as reflection of the LED irradiation, and acorneal reflection point (corneal reflection center) is detected as abright part. That is, the first calculator 2351 detects the pupil partfrom the captured image, and calculates coordinates that indicate theposition of the pupil center. The first calculator 2351 detects a regionhaving predetermined brightness or less including a darkest part in afixed region including the eye, as the pupil part, and a region havingpredetermined brightness or more including a brightest part, as thecorneal reflection. Further, the second calculator 2352 detects acorneal reflection part from the captured image, and calculatescoordinates that indicate the position of the corneal reflection center.Note that the first calculator 2351 and the second calculator 2352calculate respective coordinate values for two images obtained by theright and left cameras (step S204).

Note that the right and left cameras are subjected to camera calibrationby a stereo calibration method in order to acquire the three-dimensionalworld coordinates, and a conversion parameter is calculated. As thestereo calibration method, any conventionally used method can beapplied, such as a method using the Tsai's calibration theory or thelike.

The first calculator 2351 and the second calculator 2352 convert thecoordinates of the right and left cameras into three-dimensional worldcoordinates of the pupil center and the corneal reflection center, usingthe conversion parameter (step S205). The third calculator 2353 obtainsa straight line that connects the obtained world coordinates of thecorneal reflection center, and world coordinates of a center position ofthe LED light source 2103 (step S206). Next, the third calculator 2353calculates a straight line that connects world coordinates of a centerof the target image displayed at the one point on the display screen101, and the world coordinates of the pupil center (step S207). Thethird calculator 2353 obtains an intersection point of the straight linecalculated at step S206 and the straight line calculated at step S207,and employs this intersection point as the corneal curvature center(step S208). The third calculator 2353 calculates a distance between thepupil center and the corneal curvature center of this time, and storesthe calculated distance in the storage 150, or the like (step S209). Thestored distance is used to calculate the corneal curvature center at asubsequent time of detection of a gaze point (eye gaze).

The distance between the pupil center and the corneal curvature centerof when the subject gazes at the one point on the display 210 in thecalculation processing is constantly maintained within a range ofdetecting the gaze point in the display 210. The distance between thepupil center and the corneal curvature center may be obtained from anaverage of entire values calculated during the reproduction of thetarget image, or may be obtained from an average of values of severaltimes, of values calculated during the reproduction.

FIG. 18 is a diagram illustrating a method of calculating a position ofa corrected corneal curvature center, using the distance between thepupil center and the corneal curvature center obtained in advance, whenthe gaze point is detected. A gaze point 805 indicates the gaze pointobtained from the corneal curvature center calculated using the typicalcurvature radius value. A gaze point 806 indicates the gaze pointobtained from the corneal curvature center calculated using the distanceobtained in advance.

A pupil center 811 and a corneal reflection center 812 respectivelyindicate the position of the pupil center and the position of thecorneal reflection center calculated at the time of detecting the gazepoint. A straight line 813 is a straight line that connects the LEDlight source 2103 and the corneal reflection center 812. A cornealcurvature center 814 is the position of the corneal curvature centercalculated from the typical curvature radius value. A distance 815 isthe distance between the pupil center and the corneal curvature centercalculated in the previous calculation processing. A corneal curvaturecenter 816 is the position of the corneal curvature center calculatedusing the distance obtained in advance. The corneal curvature center 816is obtained from the facts that the corneal curvature center exists onthe straight line 813, and the distance between the pupil center and thecorneal curvature center is the distance 815. Accordingly, an eye gaze817 calculated when the typical curvature radius value is used iscorrected to an eye gaze 818. Further, the gaze point on the displayscreen 101 is corrected from the gaze point 805 to the gaze point 806.Note that connection of the right and left cameras (the right camera2202 and the left camera 2203) and the controller 2300 is notillustrated and omitted.

FIG. 19 is a flowchart illustrating an example of eye gaze detectionprocessing of the present embodiment. For example, as processing ofdetecting an eye gaze in diagnosis processing using a diagnosis image,the eye gaze detection processing of FIG. 19 can be executed. In thediagnosis processing, processing of displaying a diagnosis image,evaluation processing by the evaluator 2357 using the detection resultof the gaze point, and the like are executed, in addition to the stepsof FIG. 19.

Steps S301 to S305 are similar to steps S202 to S206 of FIG. 17, andthus descriptions are omitted.

The third calculator 2353 calculates a position on the straight linecalculated at step S305, and in which the distance from the pupil centeris equal to the distance obtained in the previous calculationprocessing, as the corneal curvature center (step S306).

The eye gaze detector 2354 obtains a vector (eye gaze vector) thatconnects the pupil center and the corneal curvature center (step S307).This vector indicates the eye gaze direction that the subject is lookingat. The gaze point detector 2355 calculates three-dimensional worldcoordinate values of an intersection point of the eye gaze direction andthe display screen 101 (step S308). The values are coordinate valuesthat express the one point on the display 210 that the subject gazes at,in the world coordinates. The gaze point detector 2355 converts theobtained three-dimensional world coordinate values into coordinatevalues (x, y) expressed in a two-dimensional coordinate system of thedisplay 210 (step S309). Accordingly, the gaze point on the display 210that the subject gazes at can be calculated.

Calculation processing of calculating the distance between the pupilcenter position and the corneal curvature center position is not limitedto the method described in FIGS. 16 and 17. Hereinafter, another exampleof calculation processing will be described using FIGS. 20 and 21.

FIG. 20 is a diagram for describing calculation processing of thepresent modification. Elements described in FIGS. 10 to 13, and 16 aredenoted with the same reference signs, and descriptions are omitted.

A line segment 1101 is a line segment (first line segment) that connectsa target position 605 and an LED light source 103. A line segment 1102is a line segment (second line segment) that is parallel to the linesegment 1101, and connects a pupil center 611 and a straight line 613.In the present modification, a distance 616 between the pupil center 611and a corneal curvature center 615 is calculated using the line segment1101 and the line segment 1102, and stored.

FIG. 21 is a flowchart illustrating an example of calculation processingof the present modification.

Steps S401 to S407 are similar to steps S201 to S207 of FIG. 17, andthus descriptions are omitted.

A third calculator 2353 calculates a line segment (the line segment 1101in FIG. 20) that connects a center of a target image displayed in onepoint on a screen of a display 210, and a center of the LED light source103, and calculates a length (L1101) of the calculated line segment(step S408).

The third calculator 2353 calculates a line segment (the line segment1102 in FIG. 20) that passes through the pupil center 611, and isparallel to the line segment calculated at step S408, and calculates alength (L1102) of the calculated line segment (step S409).

The third calculator 2353 calculates the distance 616 between the pupilcenter 611 and the corneal curvature center 615, based on the fact thata triangle having the corneal curvature center 615 as a vertex, and theline segment calculated at step S408 as a base, and a triangle havingthe corneal curvature center 615 as a vertex, and the line segmentcalculated at step S409 as a base have a similarity relationship (stepS410). For example, the third calculator 2353 calculates the distance616 such that a ratio of the length of the line segment 1102 to thelength of the line segment 1101, and a ratio of the distance 616 to adistance between the target position 605 and the corneal curvaturecenter 615 become equal.

The distance 616 can be calculated by the following formula (1). Notethat L614 is the distance from the target position 605 to the pupilcenter 611.The distance 616=(L614×L1102)/(L1101×L1102)  (1)

The third calculator 2353 stores the calculated distance 616 in astorage 150 and the like (step S411). The stored distance is used tocalculate the corneal curvature center at a subsequent time of detectionof a gaze point (eye gaze).

As described above, according to the present embodiment, effects asfollows can obtained, for example.

(1) It is not necessary to arrange the light source (illuminator) in twoplaces, and detection of an eye gaze can be performed with the lightsource arranged in one place.

(2) Because the light source is arranged in one place, the device can bemade compact, and a decrease in cost can be realized.

Although the invention has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. A diagnosis supporting device comprising: adisplay; a stereo camera configured to capture an image of a subject; aneye gaze detector configured to detect an eye gaze direction of thesubject, from the image captured by the stereo camera; a gaze pointdetector configured to detect a gaze point of the subject in a displayregion of the display based on the eye gaze direction; an outputcontroller configured to display any of a plurality of diagnosis imageson the display, select a diagnosis image, of the plurality of diagnosisimages, to be displayed next according to a position of the gaze pointdetected when the diagnosis image is displayed, and display thediagnosis image on the display; and an evaluator configured tocalculate, for the respective diagnosis images, an evaluation value ofthe subject that corresponds to a degree of a development disorder and aratio of looking at a predetermined region within the diagnosis imagebased on the gaze point detected by the gaze point detector when thediagnosis image is displayed, wherein each of the plurality of diagnosisimages belongs to one out of a plurality of groups, and the outputcontroller selects, when the diagnosis image belonging to a first group,of the plurality of groups, which includes similar diagnosis images usedto calculate a similar evaluation value is displayed, and, further, whena ratio of positions of the gaze point indicates that a number of thegaze points detected in a predetermined region in the diagnosis imageand in a predetermined period is equal to or greater than apredetermined threshold, another diagnosis image belonging to the firstgroup as the diagnosis image to be displayed next, and when the ratio ofthe positions of the gaze point indicates that the number of the gazepoint detected in the predetermined region in the diagnosis image and inthe predetermined period is less than the predetermined threshold, adiagnosis image belonging to a second group, of the plurality of groups,that is different than the first group as the diagnosis image to bedisplayed next, wherein the ratio is a ratio of the number of gazepoints detected in the predetermined region to a total number of thedetected gaze points, or a ratio of a total time in which the gaze pointis detected within the predetermined region to a total time in which thegaze point is detected.
 2. The diagnosis supporting device according toclaim 1, wherein the diagnosis image includes an image of a person andan image of a geometrical pattern, the first group includes a firstdiagnosis image and a second diagnosis image in which a position of theimage of the person and a position of the image of the geometricalpattern are symmetrical about a line that passes through a vicinity of acenter of the diagnosis image in an up and down direction, and theoutput controller selects, according to the position of the gaze pointdetected when the first diagnosis image classified into the first groupis displayed, one of the second diagnosis image or the diagnosis imageclassified in the second group as the diagnosis image to be displayednext.
 3. The diagnosis supporting device according to claim 1, furthercomprising: an illuminator including a light source that performsirradiation of light; a target position provided on the display; aposition detector configured to calculate a first position indicating acenter of a pupil and a second position indicating a center of a cornealreflection, from an image of an eye ball of the subject irradiated withthe light by the illuminator and captured by the stereo camera; and acalculator configured to calculate a third position indicating acurvature center of a cornea as an intersection point of a straight linethat connects the first position and the target position, and a straightline that connects the second position and a position of the lightsource, wherein the eye gaze detector detects an eye gaze of the subjectbased on the first position and the third position.
 4. The diagnosissupporting device according to claim 1, wherein the output controller isconfigured to, when the ratio of the positions of the gaze point notincluded in the predetermined region is equal to or greater than thanthe predetermined threshold in a last diagnosis image included in thefirst group, select a diagnosis image from the second group to bedisplayed next.
 5. A method of supporting diagnosis, the methodcomprising: an eye gaze detection step of detecting an eye gazedirection of a subject from a captured image captured by a stereo camerathat captures the subject; a gaze point detection step of detecting agaze point of the subject in a display region of a display based on theeye gaze direction; an output control step of displaying any of aplurality of diagnosis images on the display, selecting a diagnosisimage, of the plurality of diagnosis images, to be displayed nextaccording to a position of the gaze point detected when the diagnosisimage is displayed, and displaying the selected diagnosis image on thedisplay; and an evaluation step of calculating, for the respectivediagnosis images, an evaluation value of the subject that corresponds toa degree of a developmental disorder and a ratio of looking at apredetermined region within the diagnosis image based on the gaze pointdetected in the gaze point detection step when the diagnosis image isdisplayed, wherein each of the plurality of diagnosis images belongs toone of a plurality of groups, and the output control step selects, whenthe diagnosis image belonging to a first group which includes similardiagnosis images used to calculate a similar evaluation value isdisplayed, and, further, when a ratio of positions of the gaze pointindicates that a number of the gaze points detected in a predeterminedregion in the diagnosis image and in a predetermined period is equal toor greater than a predetermined threshold, another diagnosis imagebelonging to the first group as the diagnosis image to be displayednext, and when the ratio of positions of the gaze point indicates thatthe number of the gaze point detected in the predetermined region in thediagnosis image and in the predetermined period is less than thepredetermined threshold, a diagnosis image belonging to a second group,of the plurality of groups, different from the first group as thediagnosis image to be displayed next, wherein the ratio is a ratio ofthe number of gaze points detected in the predetermined region to atotal number of the detected gaze points, or a ratio of a total time inwhich the gaze point is detected within the predetermined region to atotal time in which the gaze point is detected.
 6. The method ofsupporting diagnosis according to claim 5, wherein the output controlstep selects a diagnosis image to be displayed next included in thesecond group if the ratio of the positions of the gaze point notincluded in the predetermined region is equal to or greater than thanthe predetermined threshold in a last diagnosis image included in thefirst group.